Selection methods for extended least squares support vector machines

Support vector machines (SVMs), have proven to be effective for solving learning problems, and have been successfully applied to a large number of tasks. Lately a new technique, the Least Squares SVM (LS-SVM) has been introduced. This least squares version simplifies the required computation, but sparseness –a really attractive feature of the standard SVM– is lost. To reach a sparse model, further processing steps –e.g. pruning– must be applied. These steps however increase the algorithmic complexity of the training and at the same time the quality of the results may degrade. To overcome these problems an extended version of LS-SVM has been proposed. This solution uses a special “partial reduction” technique, where the LS-SVM training is reformulated to result in a sparse but precise model that can be constructed more effectively. The reduction is based on a support vector selection method, which has a great effect on the performance of the model. Originally an automatic data selection method was proposed to determine the support vectors for the extended LS-SVM. In this paper it is shown, that existing methods can also be used in conjunction with the partial reduction method. The selection methods are analyzed, and based on simulations their performance is compared.